Supporting Collaborative Ontology Development in Protégé

Ontologies are becoming so large in their coverage that no single person or a small group of people can develop them effectively and ontology development becomes a community-based enterprise. In this paper, we discuss requirements for supporting collaborative ontology development and present Collaborative Protégé—a tool that supports many of these requirements, such as discussions integrated with ontology-editing process, chats, and annotations of changes and ontology components. We have evaluated Collaborative Protégé in the context of ontology development in an ongoing large-scale biomedical project that actively uses ontologies at the VA Palo Alto Healthcare System. Users have found the new tool effective as an environment for carrying out discussions and for recording references for the information sources and design rationale. 1 Ontology Development Becomes Collaborative Recent developments are dramatically changing the way that scientists are building ontologies. First, as ontologies are becoming commonplace within many scientific domains, such as biomedicine, they are being developed collaboratively by increasingly large groups of scientists. Second, ontologies are becoming so large in their coverage (e.g., NCI Thesaurus with 80K concepts) that no one user or small group of people can develop them effectively. Hence, organizations such as the NCI Center for Bioinformatics “outsource” some of their ontology development to the scientific community at large. Third, in the last one or two years, many users have become quite familiar and comfortable with the concept of user-contributed content, both in their personal and professional lives (cf. Web 2.0). Thus, domain experts need tools that would support collaborative ontology development and would include collaboration as an integral part of the ontology development itself. Researchers are only now beginning to develop such tools. Last year, tool developers were invited to contribute their tools for collaborative construction of structured knowledge (which included not only ontologies, but also any structured data) to the CKC Challenge, which brought together developers and users in order to examine the state-of-the-art and to understand the requirements for new tools [10]. In general, the participants in the CKC Challenge agreed on several key points. First, the notion of collaborative development of ontologies and most of the tool support was in its infancy. Second, the spectrum of tools even in the relatively small set of the challenge participants (from tools to organize tags in a hierarchy to full-fledged ontology editors) demonstrated that no single tool is likely to fill the niche completely. Third, the requirements for such tools to support collaborative development in any specific setting were still poorly understood. The challenge participants started identifying these requirements. Starting with the initial set of requirements identified as the result of the CKC workshop, we continued the requirements-gathering phase in the context of extending the Protégé ontology editor to support collaborative ontology development. To gather specific requirements, we conducted interviews with representatives of several groups that currently use Protégé for ontology development and that were trying to adopt a more formal process for development. These projects included the development of the NCI Thesaurus [17], the ontologies for the ATHENA-DSS project at the VA Palo Alto Healthcare System [7], the Ontology of Biomedical Investigations (OBI) [2], the RadLex ontology for annotating radiological images [14], and many others. As the result of this process, we collected a set of requirements for an ontology editor supporting collaboration. Note that we focused on the projects that need a full-fledged ontology editor and where ontologies are fairly rich in structure and large in size. For example, the NCI Thesaurus is an OWL DL ontology with more than 80K classes, several thousand of which are defined classes. Both RadLex and ATHENA-DSS ontologies are frame-based ontologies that use different types of constraints on properties extensively. We then developed Collaborative Protégé by extending the Protégé tool with a set of features to support these requirements. We have performed the formative evaluation of Collaborative Protégé in several different projects in order to evaluate the usability of the tool and to understand what users like and do not like about it, how they use it, and what other features they need to support their work. More specifically, this paper makes the following contributions: – We identify a set of requirements for developing expressive ontologies and knowledge bases collaboratively (Section 2). – We present Collaborative Protégé—an ontology editor that supports collaboration through integration of features such as discussions, chats, and annotations in the ontology editor (Sections 4, 5, and 6). – We perform the formative evaluation of Collaborative Protégé in the context of representing formally clinical practice guidelines in the ATHENA-DSS project (Sections 7, 8, and 9). 2 Requirements for Support of Collaborative Ontology Development We have identified our requirements for tool support for collaborative ontology development through interviews with many institutional Protégé users. The requirements that we identified significantly extend the set of requirements from the CKC workshop, and focus on the requirements of ontology developers for domains such as biomedicine. These developers are usually domain experts rather than knowledge engineers. In most of these projects, users have already used Protégé in a client–server mode that enabled distributed users to edit the ontology simultaneously, immediately seeing the changes that others make. Thus, we focused on the features that would explicitly support collaboration. Furthermore, by the nature of projects already having chosen